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1. Introduction 

A first-order language is called monadic if its non-logical vocabulary 
lacks both function symbols and relation symbols of arity greater than 
one. Formulas of this kind of language can be mechanically tested for 
validity but its expressive power seems to be too restricted to be of any 
interest for linguistics. Monadic second-order languages on the other hand 
extend the language of (full) first-order logic by providing for quantifiable 
monadic second-order variables. These languages increase the definitional 
resources of first-order logic substantially but they still retain many meta- 
mathematical properties which make them a 'good source of expressive 
and manageable theories.' ( purevich 1985 ) 



In the paper from which the preceding remark is taken Gurevich gives 
a detailed account of two strong decidability techniques which were de- 
veloped in connection with monadic second-order theories. One of these 
techniques is based on games and automata while the other employs gen- 
eralized products. The applicability of these methods to certain second- 
order theories where quantification is restricted to variables ranging over 
subsets of the domain of discourse underlines the fact that formaliza- 
tions couched in this linguistic framework are supported by a manageable 
underlying logic. In sharp contrast with the situation exemplified by 
monadic first-order logic the decidability of applied forms of its second- 
order counterpart detracts nothing from its value as a flexible expressive 
means for the presentation of theories which embody interesting parts of 
either mathematics or the empirical sciences. 

This is not the proper place to substantiate this last claim as far as 
mathematical theories are concerned. The interested reader is referred 
to the paper by Gurevich where she can find a wealth of examples. The 
development which brought linguistics into the picture was initiated by 
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Rogers (1994) who showed in his dissertation that the majority of tech- 
nical assumptions which form the core of the Principles and Parameter 
incarnation of transformational grammar are conducive to transparent 
formalization in monadic second-order tree logic. Using Rabin's second- 
order logic with multiple successors he was able to give convincing defi- 
nitions of such central notions as government, chain and category and, in 
addition to these positive results, to make a significant contribution to the 
theory-internal debate among students of the Principles and Parameter 
model by proving that such notions as free-indexing would make Rabin's 
logic undecidable when added to its vocabulary. Especially this delim- 
iting consequence constitutes a definite advantage of Roger's approach 
over competing attempts at formalizing the theoretical concepts underly- 
ing different versions of transformational grammar. The main drawback 
of these attempts consists in an excess of expressive power, which makes it 
extremely difficult to establish any formal criteria to separate the empiri- 
cally fruitful from the spurious constructs in the Principles and Parameter 
model. 

Unfortunately, there remains one difficulty which threatens to obstruct 
the project of using monadic second-order logic as the proper framework 
for a restrictive linguistic metatheory. This difficulty has its origin in 
the close relationship between recognizable tree sets and weak monadic 
second-order logic of multiple successors. It has been known since the 
pioneering work of Doner (1970) and Thatcher and Wright (1968) that 
the sets of finite trees which can be generated by regular tree grammars 
are identifiable with the collections of finite trees definable by formulae 
of the weak version of Rabin's logic. Since the corresponding string sets, 
i.e. the concatenation of the labels that decorate the leaves of trees in 
either of these equivalent classes, are exactly identical to the context-free 
languages the question of whether monadic second-order logic of multiple 
successors provides an adequate formal framework for linguistic purposes 
is inseparable from the issue of the proper location of natural languages 
on the Chomskyan hierarchy. E vidence from lan guages like Swiss- German 
( Huybregts 1984 , ghieber 1985 ) and Bambara ( Culy 1985 ) suggests that 
natural languages exhibit a certain degree of context-sensitivity and that 
a formal system which aims at serving as an adequate mathematical basis 
for the working linguist must accommodate a corresponding amount of 
structural complexity. 

There are serious obstacles, however, to any program devoted to a 
strengthening of Rabin's logic. As has been shown by several people, even 
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minor additions to the vocabulary of the underlying language lead to un- 
decidability (cf. Lauchli and Savioz 1987). The set of theorems of monadic 
tree theory, another designation of the theory of multiple successors which 
refers to its intended model, seems to be of so high a degree of complexity 
that enriching its language always is in danger of overshooting the mark. 
A natural alternative to an extension of the basic vocabulary of the lan- 
guage would be a restricted form of second-order polyadic quantification. 
In such a move one would allow relational variables into the language, 
together with quantifiers binding them. In sharp contrast to the monadic 
set variables which can be assigned an arbitrary subset of the universe 
of discourse as a value, the range of interpretation of these new variables 
would have to be narrowed to special relations over the universe of dis- 
course. Earlier work on the monadic theory lets one expect that a careful 
restriction of the quantificational realm along the indicated lines could 
generate the desired supplementary expressiveness without losing control 
over the class of theorems. Of particular significance for the current issue 
are the results by Rabin (1972 ) and Siefkes (1975 ) to the effect that re- 
stricting quantification to regular subsets leads to a proper subsystem of 
the logic, whereas restricting quantification to recursive subsets leaves the 
class of theorems untouched. The kind of relations in the polyadic case 
one will have to consider should perhaps not be determined with refer- 
ence to their computational complexity, but rather with reference to their 
internal structure. The recent characterization of context-free languages 
as sets of strings which can be defined by first-order sentences with an 
initial second-order existential quantifier provides an instructive example 
for such a semantic approach in a related context. Extending the work 
of Biichi (1960) and Elgot (1961) Lautemann et al. (1995) have shown 
that the model sets satisfying sentences of dyadic second-order logic 
of one successor coincide with the context-free languages if the existen- 
tial second-order quantifier ranges over binary relations that display the 
dependencies of matching parentheses. It is an open problem whether 
the introduction of similarly restricted dyadic quantifiers into the tree 
language with multiple successors would be accompanied by a controlled 
increase of the definitional resources and in this way vindicate the idea of 
a secondary semantics for the limited amount of context-sensitivity that 
is exemplified by natural languages. 

We cannot end this short outline of possible ways out of the narrow 
confines of context-freeness without mentioning free variable second-order 
logic. This logic is expressed in a language in which relation and func- 
tion variables have free, but no bound occurrences. What this means for 
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our monadic second-order tree language is that set variables may still be 
syntactically bound by higher-order quantifiers but the free relation and 
function variables get an interpretation which treats them implicitly as 
universally quantified. Again, it appears to be an open question whether 
this extension of syntactic resources steers clear of undecidability. 

Even though one of the sketched strengthenings of the underlying 
tree logic — extension of the non-logical vocabulary, restricted polyadic 
second-order quantification or the introduction of free function and re- 
lation variables — may secure just the right increase in expressive power, 
the precarious status of the monadic second-order theory of multiple suc- 
cessors verging on the undecidable gives little cause for hope. We shall 
therefore propose a hybrid solution to the problem of how to account 
for mildly context-sensitive phenomena with the help of tree logic. The 
limited expressive power of this logic in its original set-up makes it im- 
possible to formulate the solution in a way that would deal directly with 
the problematic phenomena, but we can give these phenomena a slightly 
different appearance whereby they do become context-free and as such 
definable in tree logic. 

To be somewhat more specific, we wi ll use an "alge braicized" variant of 
grammars with macro-like productions (Fischer 196S) to present an analy- 
sis of structural dependencies which are beyond the reach of context-free 
devices. Macro grammars constitute a natural extension of traditional 
rewrite systems. Whereas in traditional presentations of rule systems for 
abstract language families the emphasis has been on a first-order sub- 
stitution process in which auxiliary variables are replaced by elements of 
the carrier of the proper algebra — concatenations of terminal symbols and 
auxiliary category variables in the string case — macro grammars lift this 
process to the second-order level of operations defined on the elements of 
the carrier of the algebra. Once this second-order substitution process is 
turned into a process involving only individual variables we are back on 
familiar ground, the only difference being the shift from single-sorted to 
many-sorted algebras. 

When, two paragraphs back, we spoke about two different representa- 
tions of context-sensitive dependencies we had this shift from a functional 
to an objectual substitution process in mind. What makes the transfor- 
mation of the higher-order into an objectual substitution possible is the 
technical device of a derived or lifted alphabet. In informal terms, the 
formation of a derived alphabet can be explained as a nominalization 
process turning functions into objects. As a result, these latter saturated 
entities have to rely on an explicit composition operation that provides 
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the necessary glue for combining the objectual counterpart of the origi- 
nal functional entity with its arguments. Since the objectual counterpart 
now belongs to the type of saturated entities, the substitution process 
on the symbolic level, when applied to the nominalized expressions re- 
sulting from the derivation of the original alphabet, can then take the 
familiar form of first-order substitution. As should be obvious from the 
preceding remarks, the language generated by this first-order substitu- 
tion procedure is context-free and as such amenable to logical analysis 
in the framework of monadic second-order logic with multiple successors. 
There is, however, a negative aspect of the approach using derived alpha- 
bets. During the translation of the original alphabet into its nominalized 
derived counterpart new operators representing functional composition — 
and, furthermore, new constants standing for projection functions which 
we have suppressed in our informal account — are introduced that change 
the configurational shape of the translated expressions. This point can 
be made more vivid in terms of a notational format that is taken for il- 
lustrative purposes from lambda calculus with parallel abstraction. Let 
{xi, . . . Xn) denote parallel (n-ary) abstraction and comp(-,-) binary ap- 
plication. The lifted variant of the term /(a, 6) then corresponds to the 
following expression comp{{xi,X2)f{xi,X2),p{a,b)), where p represents 
pair formation in the derived alphabet. This example has to be taken 
with a grain of salt because in the parallel lambda calculus both the 
redex and its converted form are supposed to have the same meaning 
whereas in the context of formal language theory the simple fact that 
these expressions are different complex symbols embodies two competing 
proposals for syntactic analysis. 

In the last section of the paper we show how to solve the tension be- 
tween the definitions for linguistic concepts in terms of structural config- 
urations of the original trees and their counterparts in the lifted context. 
While the definitions that are informed by the original set-up are exten- 
sionally inadequate in the general case — they fail to refer to the context- 
sensitive dependencies — their lifted counterparts, living in an environment 
of first-order substitution, can be combined with adequate characteriza- 
tions of those context-sensitive structures. This combination is made 
possible by the closure of call by value tree languages under determinist ic 
bottom-up tree transducer mappings (see Engelfriet and Schmidt 197^ ). 

We have not attempted to present a defense of macro grammars or 
of a theory of structural notions embodied in this particular format. A 
detailed discussion of the (de-)merits of macrogrammatical analyses of a 
range of syntactic problems is contained in H.-P. Kolb's contribution to 
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the present volume. It is worth emphasizing that our apphcation of tree 
theory to context-sensitive structures is not intended as a justification for 
a particular form of syntactic analysis. This task remains to be done and 
we would be delighted if others investigated the structural restrictions 
that characterize a program of (derived) syntactic macrotheory. 

There are several sources that have influenced the ideas reported here. 
Apart from the work on the logical characterization of language classes 
that was mentioned above the development in universal algebra that led 
to a uniform, signature free treatment of varieties has been our main 
inspiration. From the very beginning of this development it has been 
obvious to people working in this field that closed sets of operations are 
best presented as categories with finite products. When this presentation 
is retranslated into the language of universal algebra we are confronted 
with a signature whose only non-constant operators are symbols for target 
tupling and functional composition. Algebras with signatures of this type 
will play a major role in the paper and they will provide the technical 
foundation for extending the logical methods of descriptive complexity 
theory to context-sensitive phenomena. 

The first to see the potential for tree language theory of this type of 
lifted signature was Maibaum (1974). He showed in particular how to map 
context-free into regular tree productions rules. Unfortunately, a substan- 
tial part of his results are wrong because he mistakenly assumed that for 
so called call by name derivations an unrestricted derivation in a lifted 
signature would leave the generated language unchanged. Engelfriet and 



Schmidt (1977, 1978) point out this mistake and give fixed-point charac- 



terizations for both call by name and call by value context-free production 
systems. We hope that the present paper complements the denotational 
semantics for call by name tree languages by giving an operational anal- 
ysis of the derivation process both on the level of the original and on the 
level of the lifted signature. 



2. Preliminaries 

The purpose of this section is to fix notations and to present definitions 
for the basic notions related to universal algebra. The key notion is that 
of a derived algebra. As was indicated above, the realization that derived 
algebras allow for a different presentation based on a lifted signature con- 
stitutes the main step in the process of restoring context-freeness. In this 
context, we will be involved in a generalization of formal language con- 
cepts. Many-sorted or heterogeneous algebras provide the proper formal 
environment to express this generalization. For expository purposes we 
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will not give every definition in its most general form but keep to the 
single-sorted case where the reader can easily construct the extension to 
a situation where more than one sort matter. 

Definition 1. Lot S be a set of sorts (categories). A many— sorted 
alphabet S is an indexed family (S^^^ \ w G S* , s € S) of disjoint sets. 
A symbol in Eu,.., is called an operator of type {w,s), arity w, sort s 
and rank l(w). If w = e then / G Sg^^ is called a constant of sort s. 

l{w) denotes the length of ru. 

Note that a ranked alphabet in the traditional terminology can be 
identified with an ^-sorted alphabet where S = {s}. The set Egr. ^ is 
then the same as S„. In the way of explanation, let us print out that 
each symbol / in Su,^^, represents an operation taking n arguments, the 
ith argument being of sort Wi, and yielding an element of sort s, where 
w = Wi ■ ■ ■ Wn ■ Alternative designation for many-sorted alphabets are 
many-sorted signatures or many-sorted operator domains. We list some 
familiar examples of single-sorted signatures for further reference. 

Example 1. 

a) So = {e}UV E2 = {-} 

Single-sorted signature of semi-groups, extended by a finite set of 
constants V. 

b) So = {£} Si = {a I a e V} 
Single-sorted signature of a monadic algebra. 

c) S2 = {A,V} 

Single-sorted signature of lattices. 

As was mentioned above, a full description of the theory of a class of 
algebras of the same similarity type is given by the totality of the derived 
operations. These operations can be indicated by suitably constructed 
terms over the basic operators and a set of variables. 

Definition 2. For a many-sorted alphabet S, we denote by T(S) the 
family (T(S, s) \ s € S) oi trees of sort s over S. T(S, s) is inductively 
defined as follows: 

(i) For each sort s & S 

S,,, C T(S, s) 

(ii) For n > 1 and s € S, w € S* , if f € S^^^ and for 1 < i < n, 
ti e r(S, Wi), l{w) = n, 
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Definition 3. For a many sorted alphabet S and a family of disjoint 
sets Y = (Ys\sG S), the family r(S, Y) is defined to be T(S(y)), where 
is the many-sorted alphabet with I]{Y)^ s = ^ U Yg and for w ^ 
e, T,{Y)yj^s = Tiyj^s We call a subset C of T{T,, s) a tree language over 
S (of sort s). 

Having described the syntax of the tree terms and having indicated 

their intended interpretation, it remains to specify the central notion of 
an algebra and to give a precise definition of the way in which the formal 
term symbols induce an operation on an algebra. 

Definition 4. Suppose that 5 is a set of sorts and that S is a many- 
sorted alphabet. A S-algebra 21 consists of an ^-indexed family of sets 
A = {As)ses and for each operator a G ^w,8, a function era : — > 
where = A^^ x • • • x A^" and w = 1^1- ■ ■ w„. 

The family A is called the sorted carrier of the algebra 21 and is 
sometimes written |2l|. 

Different algebras, defined over the same operator domain, are related 
to each other if there exists a mapping between their carriers that is 
compatible with the basic structural operations. 

Definition 5. A E-homomorphism of S-algebras h : 21 — > *B is an 
indexed family of functions hs : Ag — > Bs, {s € S) such that for every 
operator a of type {w, s) 

/is(o-2i(ai, . . . ,a„)) = (TfB{hwi{ai), ■ ■ ■,hw„{an)) 
for every n-tuple (ai, . . . , a„) G A^ . 

The set of sorted trees T(S, Y) can be made into a S-algebra by defin- 
ing the operations in the following way. For every a in S^_s, for every 

0'l(S,y)(ilj ■ ■ ■ itn) — ^(tl, . . . , 

If 21 is a S-algebra, each term t e T{T,{Yu,)^ s) induces a function 

ta : A"' A' 

called a derived operation. In the place of Y we have used the set of 
sorted variables Yyj := {yi^wi 1 1 < « < K'*^)}- The meaning of the derived 
operation ta is defined as follows: for (ai, . . . , a„) G A^ 

t^{ai,. ..,an)= a{t) 

where a : 1(S, Yyj) — > 21 is the unique homomorphism with a(yi,u,J = ai. 



ON CLONING CONTEXT-FREENESS 



9 



In intuitive terms, the evaluation of a S(y)-tree t in a given S-algebra 
proceeds as follows. First, one assigns a value a.{yi) G A to every variable 
in Y. Then the operations of 21 are applied to these elements of A as di- 
rected by the structure of t. There is, though, another description of the 
action performed by the derived operations on an algebra 2t. According 
to this conception the derived operations of an algebra 2t arc the map- 
pings one gets from the projections ygi by iterated composition with the 
primitive operations era (cr G S). 

Given any S-algcbra 21 we can describe the process of determining the 
set of derived operations within the framework of another algebra that is 
based on a different signature. In this new signature the symbols of the 
original alphabet S are now treated as constants, as rtq the projections. 
The only operations of non-zero arity are the composition functions and 
the functions of target tupling. By composition of operations is meant 
the construction of an operation h of type (w, s) from given operations / 
of type {v,s) and gi of type (w^Vi) where {w, s), {v, s), {w,Vi) ^ S* x S. 
The constructed operation h satisfies the rule 

K"') = f{9i{a),---,9k{a)) 

where k = \v\ and a E . 

If the operations of target tupling are among the basic operations that 
are denoted by the symbols of the new signature, the type of composition 
operations can be simplified to ((f,s), {w,v)), {w,s). Take again the op- 
erations gi with their types as just indicated. By their target tupling is 
meant the construction of an operation h of type {w, v) that satisfies the 
rule 

/i(a) = i9i{a),---,9k{a)) 

where again a G and the outer parentheses on the right-hand side 
indicate the ordered /e-tuple of values. Having introduced composition and 
target tupling and having indicated their intended interpretation the only 
missing ingredient that remains for us to introduce, before we can define 
the concept behind the title of the paper, in the collection of projection 
operations. 

The projection operations on a Cartesian product A^ are the trivial 
operations tt™. satisfying 

T^wi (a) = a-wi 

A closed set of operations or, more briefly, a clone on a family of non- 
void sets A = (As) (s G S) is a set of operations on A that contains the 
projection operations and is closed under all compositions. The relevance 



10 



UWE MONNICH 



of clones for the purposes of understanding the hidden structure of an 
algebra and of providing a signature free treatment for universal algebra 
was first realized by P. Hall. An alternative proposal under the name of 
algebraic theories and their algebras is due to J. Bcnabou and W. Lawvcre. 
We have chosen to follow the example of J. Gallier and of J. Engelfriet and 
E. M. Schmidt in using the style of presentation familiar from standard 
universal algebra. Our only departure from this tradition is the explicit 
inclusion of the operation of target tupling into an official definition of 
the clone of term operations. 

Definition 6. The clone of term operations of an S-algebra 21, de- 
noted by C\o{A), is the smallest set of operations on the carrier A — {A^) 
that contains the primitive operations of 2t, the projections and is closed 
under composition and target tupling. The set of all operations of type 
{w,s) in CIo(j4) is denoted by CIo^^.s)(j4). 

To actually present a clone as an algebra both the set of sorts S and 
the underlying alphabet S have to be adapted to the new situation. The 

idea to characterize this new situation with the help of a derived alphabet 
was first used by J. Bcnabou and later rediscovered by T. S. E. Maibauin. 

Definition 7. Let 5* be a set of sorts and S = (Suj.s) be an S'-sorted 
alphabet {{w,s) G S* x S). The derived {S* x 5*)-sorted alphabet of 
S denoted by Z)(S), is defined as follows: 



where w & S* , v G and s € S. Each tt^ is a projection operator 
of type (e, {v,Vi)), each ( )w,v is a tuphng operator and each Sv,w,s is a 
substitution or composition operator of types {{w, vi) ■ ■ ■ {w, Vn), {w, v)), 
respectively {{v, s), (w, v), {w. s)) and each a in T,ui,s becomes a constant 
operator in the derived alphabet of type (e, {w, s)). 

Example 2. We have seen above that trees form the sorted carrier of the 
S-algcbra T(S,y). What is of fundamental importance for the further 
development is the fact that trees with variables can also be seen as a 
£>(S)-algebra. The tree substitution algebra T)1{Y) is a D(E)-algebra 
whose carrier of sort {w,s) is the set of trees T{Y,,Yui)s, i.e. the set of 
trees of sort ,s that may contain variables in In order to alleviate our 
notation we will denote this carrier by T(w, s). Carriers of sort (w, v) are 
t;-tuples of carriers of sort {w, Vi) and are denoted by T{'w, v). Each a in 
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^w,s is interpreted as the tree crit/i^w-L^ • • • ^Un^wn 

), each TTj" is interpreted 

as yi,vii each ( ).w,v{tii ■ ■ ■ ,tn) is interpreted as the formation of t;-tuples 
where each ti is an element of T{w,Vi) and l{v) = fc, and 
each Sy^uj^s is interpreted as a composition or substitution of trees. An 
intuitive description of the composite Sy^w,ss.^(^ Y)i^''^') ^^^^ t G T{v,s) 
and t' = {ti, . . . ,tk) & T{w, v) is this: the composite is the term of type 
(ly, s) that is obtained from t by substituting the term ti of sort Vi for the 
variable yi^vi in t- The formal definition of the composition operation relies 
on the unique homomorphism r : 'r(E, Y^) — > 'X(E, Y^) that extends the 
function t' : Yy — > Ui'^(^;^i) mapping yi^y. to ti in T{w,Vi). Then for 
any t G T{v, s), we define Sy^w,s^^^^ y) as the value of t' on the term 
t: 

^v,w,S'Q^(^-^ Y) ^ ) ■ (^) ^[^1? ■ • ■ 7 ^n] 

where the last term indicates the result of substituting ti for yi^y-. 

Since the derived alphabet 0(1]) leads to a tree algebra 1{D{T,)) in 
the same way that the alphabet S led to the algebra 1(1]), there is a 
unique homom orphism /3 : 1{D{Y,)) — > S)T(E). It was pointed out by 



Gallicr (1984 ) that this homomorphism is very similar to /3-conversion in 
the A-calculus. The explicit specification of its action repeats in a concise 
form the description of the tree substitution algebra: 

0w.s{cr) = a-{yi^y,^,...,yn,w„) for a G E^ 



Pw.siT^T) = yi.w, if Wi = S 

Pw, sii )wA,{ti, ■■■ ,tk) = {ti,...,tk) for ti eT{w,Vi) 

for t e T{v,s), ti £ T{w,Vi) 
and t' = (ti,.. .,tk). 

Example 3. Suppose that the set of sorts 5 is a singleton and that S 
contains three symbols /, a and b where T,ss.s — {/} and ^e,s = {oj 
and s is the single sort in S. As is customary in the context of single- 
sorted alphabets we shall write the type (s", s) as n. According to this 
notational convention the following figure displays a tree t in T(Z)(E)): 

'S'2.0 
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Applying /3q to this tree it returns as a value the tree Po{t) = f{a,b) 
in T'(S). Displayed in tree form this last term looks as follows: 

/ 




The unique homomorphism jS from the derived tree algebra into the 
tree substitution algebra has a right inverse LIFT^ = (LIFTj g){{w,s) G 
S* X S), where LIFT^ ^, maps a S-tree in r(i;(Ft„),s) into a £>(E)-tree 
in T(D(S), {e, {w, s))). Since we will have no occasion to apply the func- 
tion LIFT to trees over a many-sorted alphabet we content ourselves with 
giving its recursive definitions for the single-sorted case: 

Definition 8. Suppose that S is a single-sorted or ranked alphabet. For 
fc > 0, LIFTf : T(E,Xfe) — > T(D(S), (s, fc)) is the function defined 
recursively as follows (where Xk = {xj 1 1 < i < k}): 

LIFTf(x.) = 

LIFTf(a) = So,k,ii<y)iova€i:o 
LIFT|=(<7(ti,...,i„)) = 5n,M(^,( kn>(LIFT^(ti),...,LIFT^(f„)) 

for a G E„. 

It should be obvious that for any tree t in T(S, Xk) 

Pk{\-\fTf{t))=t. 

The reader will have noticed that the tupling operator was conspicuous 
by its absence in the recursive definition of the LIFT-function. According 
to our official definition for the carrier of the tree substitution algebra a 
term like cr(ii, . . . , t„) is the result of composing the n-tuple ti, . . . ,t„ of 
fc-ary terms ti with the n-ary term a{xi, . . . , This part of the "struc- 
tural history" is suppressed in the third clause of the LIFT-specification. 
We shall adhere to this policy of eliminating this one layer in the config- 
urational set-up of tree terms and we shall extend this policy to the level 
of explicit trees over the derived alphabet D{I]). This does not mean 
that we revoke the official, pedantic definition of the symbol sot in a de- 
rived alphabet, but we shall make our strategy of notational alleviation 
type consistent by reading the type of each substitution operator Sy^w,s 
as {{v, s){w, vi) ■ ■ ■ {w,Vn), {w, s)) instead of {{v, s){w,v), {w, s)). 

3. Context-Free Tree Languages 

The correspondence between trees in explicit form, displaying compo- 
sition and projection labels, and their converted images as elements of a 
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tree substitution algebra is an example of a situation which is character- 
ized by a meaning preserving relationship between two algebras 21 and 
03. Of particular interest to formal language theory is the situation where 
problems in an algebra *B can be lifted to the tree level, solved there 
and, taking advantage of the fact that trees can be regarded as denoting 
elements in an arbitrary algebra, projected back into their original habi- 
tat *B. This transfer of problems to the symbolic level would, of course, 
produce only notational variants as long as the lifted environment con- 
stitutes just an isomorphic copy of the domain in relation to which the 
problems were first formulated. One might suspect that /?-convcrsion and 
its right inverse arc a case of this type. Despite their suspicious similarity, 
trees in explicit form and their cousins which arc the results of performing 
the operations according to the instructions suggested by the composition 
and projection symbols, are sufficiently different to make the transfer of 
problems to the explicit variants a worthwhile exercise. 

In intuitive terms, the difference between the two tree algebras is re- 
lated to the difference between a first-order and a second-order substitu- 
tion process in production systems. Let us view grammars as a mechanism 
in which local transformations on trees can be performed in a precise way. 
The central ingredient of a grammar is a finite set of productions, where 
each production is a pair of trees. Such a set of productions determines a 
binary relation on trees such that two trees t and t' stand in that relation 
if t' is the result of removing in t an occurrence of a first component in 
a production pair and replacing it by the second component of the same 
pair. The simplest type of such a replacement is defined by a produc- 
tion that specifies the substitution of a single-node tree to by another 
tree ti. Two trees t and t' satisfy the relation determined by this sim- 
ple production if the tree t' differs from the tree t in having a subtree ti 
that is rooted at an occurrence of a leaf node to in t. In slightly different 
terminology, productions of this kind incorporate instructions to rewrite 
auxiliary variables as a complex symbol that, autonomously, stands for 
an element of a tree algebra. As long as the carrier of a tree algebra is 
made of constant tree terms the process whereby miliary variables are 
replaced by trees is analogous to what happens in string languages when 
a nonterminal auxiliary symbol is rewritten as a string of terminal and 
non-terminal symbols, independently of the context in which it occurs. 
The situation changes dramatically if the carrier of the algebra is made 
of symbolic counterparts of derived operations and the variables in pro- 
duction rules range over such second-level entities. As we have seen in 
the preceding sections, the tree substitution algebra provides an example 
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for an algebra with this structure. The following example illustrates the 
gain in generative power to be expected from production systems deter- 
mining relations among trees that derive from second-order substitution 
of operators rather than constants. 

Example 4. Let be a finite vocabulary. It gives rise to a monadic 
signature S if all the members of V are assigned rank one and a new 
symbol s is added as the single constant of rank zero. For concreteness, 
let us assume that we are dealing with a vocabulary V that contains 
the symbols a and b as its only members. Trees over the associated 
monadic signature S = Eq U Si where Sq = {s} and Ei = {a, 6} are 
arbitrary sequences of applications of the operators a and b to the constant 
£. It is well known, as was pointed out above, that there is a unique 
homomorphism from these trees, considered as the carrier of a E-algebra 
to any other algebra of the same similarity type. In particular, there is a 
homomorphism into V* when a and b are interpreted as left-concatenation 
with the symbol a and b, respectively, and when e is interpreted as the 
constant string of length zero. This homomorphism establishes a bijection 



between T(E) and V* (cf. Maibaum 1974). When combined with this 



bijective correspondence the following regular grammar generates the set 
of all finite strings over V. 



G^{E,T, S,P) 

So = M Si = {a,b} 

To = {S} = for n > 1 

P={S ^e\a{S)\b{S)} 

C{G, S) ^ E* 

where we have identified T(E) with E*. 

P stands for the finite set of productions and s stands for the only non- 
terminal symbol. V gives also rise to a binary signature E' if the members 
of V are assigned rank zero and two new symbols are added, e of rank 
zero and ^ of rank two. Trees over this signature are nonassociative 
arc-links between the symbols a and b. When a and b are interpreted 
as constant strings of length one, e as constant string of length zero and 
the arc ^ as (associative) concatenation V* becomes an E'-algebra. Note 
that the unique homomorphism from T(E') to V* is not a bijection this 
time. When combined with this homomorphism the following grammar 
generates the string language {a"6"}. 
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G' = {E',jr,5,P') 

= {s,a,b} = {-} 

-Fo = {S} 

P' = {S^e\-{a,-{S,b))} 

C{G',S)^{'-{a,-{...,-{e,b)...b)...)} 

where n occurrences of a precede the same number of occurrences of b for 
n > 0. 

Now, consider the _D(E)-trcc substitution algebra ST(E, X). As will 
be recalled, its carrier consists of trees in T{T,,X). Due to the fact that 
E is a monadic signature trees in T(E,X) may not contain more that a 
single variable. Auxiliary symbols ranging over this carrier take therefore 
as values monadic or constant derived operators. When monadic auxiliary 
symbols appear in productions this means that they behave the way that 
nuUary auxiliary symbols do except for the fact that their argument has 
to be inserted into the unique variable slot of their replacing derived 
operator. After these explanations it should be obvious that the following 
grammar over the original monadic signature E generates the context-free 
string language {a" 6"} when combined with the unique homomorphism 
mentioned above: 



G" = (E,.F",5,P") 

■n' = {s} n = {F} 

„_ r 5 ^ F{s)\s, \ 

^ -\ F{x) - a{F{b{x)))\a{b{x)) / 

n n 

£(G",5) = {^K"^6(fe^(e)...)} 

The last grammar in the preceding example illustrates for the language 
{a"6"} a transformation that can be applied to the grammar of any given 
context-free language. Employing terminology to be introduced in a mo- 
ment, we have the precise characterization of the gain in generative ca- 
pacity resulting from the introduction of monadic operator variables into 
production systems: For every finite alphabet V, the context-free subsets 
of T{T,v) (where Ey is the monadic signature induced by V) have exactly 
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the context-free string languages as values of the unique homomorphism 
that maps trees in T(S]y) into strings in V* and interprets the elements of 
V as unary operations that concatenate for the left with their argument. 

We now turn to introducing the notion of a context-free tree grammar. 
This type of grammar is related to the type of grammars that were defined 
by Fischer (196§| ) and were called macro grammars by him. Context-free 



tree grammars constitute an algebraic generalization of macro grammars 
since the use of macro-like productions served the purpose of making 
simultaneous string copying a primitive operation. 

Definition 9. A context free tree grammar G = (E, !F, S, P) is a 4- 

tuple, where S is a finite ranked alphabet of terminals, J-' is a finite ranked 
alphabet of nonterminals disjoint from T,, S £ T is the start symbol of 
rank and P is a finite set of rules of the form 

F{xi,...,Xjn) {n>0) 

where F S and t G U JF, X„i). Recall that X is here assumed to 
be a set of variables X — {xi,X2, ■ ■ ■} and X„i — {xi, . . . , Xm}- 

For reasons having to do with the impossibility of mirroring the process 
of copying in a grammar with a completely uncontrolled derivation regime 
we restrict ourselves to one particular mode of derivation. According to 
this mode a function symbol may be replaced only if all its arguments are 
trees over the terminal alphabet. In the conventional case this form of 
replacement mechanism would correspond to a rightmost derivation. In 
terms of modes of computations, the exclusion of nonterminals that have 
any nonterminals appearing in any of their arguments corresponds to a 
call by value computation where the actual parameters of a function call 
have to be values from the domain of computation. 

Definition 10. Let G = (E, T, S, P) be a context-free tree grammar and 
let t,t' G r(E IJ J-). t' is directly derivable by an inside-out step 

from t {t ^ t') if there is a tree G r(E {J J^^Xi) containing exactly 
one occurrence of xi, a corresponding rule F{xi, . . . , Xm) — > i" ^ and trees 
h,. . . ,tm. e T(E) such that 

t = to[F{h,...,t^)] 

t' — to[t"[ti, . . . , tm]] 

t' is obtained from t by replacing an occurrence of a subtree F{ti, . . . , tm) 
by the tree t"[ti, . . . , tm]- By the inside-out restriction on the derivation 
scheme it is required that the trees ti, t2 through i„ be terminal trees. 
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Recall from the preceding section that for m, n > 0, t G T{T,, X^) and 
ti, - ■ ■ ,tm G T{J^, Xn) t{ti, . . . , tm] denotes the result of substituting ti 
for Xi in t. Observe that t[ti, . . . , tm] is in T(E, Xn). 

As is customary 4> denotes the transitive-reflexive closure of =>. 

Definition 11. Suppose G = (E, S, P) is a context-free tree grammar. 
We call 

5) = {t e T(S) I 5 4> t} 

the context-free inside-out tree language generated by G from S. 

We reserve a special definition for the case where contains only 
function symbols of rank zero. 

Definition 12. A regular tree grammar is a tuple G = {Y,, S, P), 
where S is a finite ranked alphabet of terminals, .F is a finite alphabet of 
function or nonterminal symbols of rank zero, S € is the start symbol 
and P C X T(E U .7^) is a finite set of productions. The regular tree 
language generated by G is 

£. = {tG T(S) t} 

Note that in the case of regular grammars the analogy with the con- 
ventional string theory goes through. There is an equivalence of the unre- 
stricted, the rightmost and the leftmost derivation modes where the terms 
'rightmost' and 'leftmost' are to be understood with respect to the linear 
order of the leaves forming the frontier of a tree in a derivation step. 

Very early in the development of (regular) tree grammars it was realized 
that there exists a close relationship between the families of trees gener- 
ated by tree grammars and the family of context-free string languages. 
This fundamental fact is best described by looking at it from the per- 
spective on trees that views them as symbolic representations of values in 
arbitrary domains. Recall the unique homomorphism from the introduc- 
tory example of this section that mapped non-associative concatenation 
terms into strings of their miliary constituents. This homomorphism is a 
particular case of a mapping than can easily be specified for an arbitrary 
signature. 

Definition 13. Suppose S is multi-sorted or ranked alphabet. We call 
yield or frontier the unique homomorphism y that interprets every op- 
erator in Tiyj or E„ with l{iu) = n as the n-ary operation of concatenation. 
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More precisely 

y{a) = a for cr G (or Eq) 

y{(T{ti,. . . ,tn)) = y{t\)...y{tn) for cr e (or En) and 

U e T(E)^. (or r(s)) 

Fact A (string) language is context-free iff it is the yield of a regular tree 
language. 

As was shown in the introductory example, the addition of macro op- 
erator variables increases the generative power of context-free tree gram- 
mars over monadic alphabets considerably. The following example demon- 
strates that the addition of n-ary macro operator variables leads to a sig- 
nificant extension with respect to arbitrary ranked alphabets. The string 
language of the following context-free tree language is not context-free. 

Example 5. Let us consider a context-free tree grammar G = {T,,J^,S, P) 
such that its frontier is the set of all cross-dependencies between the sym- 
bols a,c and 5, d, respectively. The grammar G consists of the components 
as shown below: 



So 

^0 



{e, a, 6, c, d} 
{S] 



S2 



{-} 

{F} 



C{G,S) = 



S — » F{a,e,c,s)\F{e,b,s,d)\e, 

F{xi,X2,X3,X4,) 

F{^{a, xi),X2, ^{c, xa), X4) | 

F{xi,^(b,X2),X3,^{d,X4)) I 
'^{'^{'^{Xi,X2),X3),X4) 



{-(-(-(-(a,... )...),-(6,...)...),-(c,...)...),-(rf.... )...))} 

The number of occurrences of a's and c's and of 6's and d's, respectively, 
has to be the same. By taking the frontier of the tree terms, we get the 
language £' = {a^b^c^d™}. 

The language of the preceding example illustrates a structure that can 
actually be shown to exist in natural language. Take the following sen- 
tences which we have taken from Shieber's (1985) paper: 

Example 6. 

(i) a) Jan sdit das mer em Hans es huus hdlfed aastriiche. 
b) John said that we helped Hans (to) paint the house. 
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(ii) a) Jan sdit das mer d'chind em Hans es huus land hdlfed aastri- 
iche. 

b) John said that we let the children help Hans paint the house. 

The NP's and the V's of which the NP's are objects occur in cross- 
serial order. D'chind is the object of land, em Hans is the object of hdlfe, 
and es huus is the object of aastriiche. Furthermore the verbs mark their 
objects for case: hdlfe requires dative case, while Idnd and aastriiche re- 
quire the accusative. It appears that there are no limits on the length of 
such constructions in grammatical sentences of Swiss German. This fact 
alone would not suffice to prove that Swiss German is not a context-free 
string language. It could still be the case that Swiss German in toto is 
context-free even though it subsumes an isolable context-sensitive frag- 
ment. Relying on the closure of context-free languages under intersection 
with regular languages Huybrcgts (1984[ ) and Bhieber (1985) are able to 



show that not only the fragment exhibiting the cross-dependencies but 
the whole of Swiss German has to be assumed as non context-free. 

Shieber intersects Swiss German with the regular language given in 
Example |^ in (iiia) to obtain the result in ([^. As is well known, this 



language is not context-free. 
Example 7. 

(iii) a) Jan sdit das mer (d'chind)* (em Hans)* hdnd wele (laa)* (hdlfe)* 

aastriiche. 

b) John said that we (the children)* (Hans)* the house wanted to 
(let)* (help)* paint. 

(iv) Jan sdit das mer (d'chind)"' (em Hans)"^ hdnd wele (laa)" (hdlfe)"^ 
aastriiche. 



Swiss German is not an isolated case that one could try to sidestep 
and to classify as methodologically insignificant. During the last 15 years 
a core of structural phenomena has been found in genetically and typo- 
logically unrelated languages that leaves no alternative to reverting to 
grammatical formalisms whose generative power exceeds that of context- 
free grammars. 

It has to be admitted that the use of macro-like productions is not the 
only device that has been employed for the purpose of providing grammar 
formalisms with a controlled increase of generative capacity. Alternative 
systems that were developed for the same purpose are e.g. tree adjoining 
grammars, head grammars and linear indexed grammars. Although these 
systems make highly restrictive claims about natural language structure 
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their predictive power is closely tied to the individual strategy they ex- 
ploit to extend the context-free paradigm. The great advantage of the tree 
oriented formalism derives from its connection with descriptive complex- 
ity theory. Tree properties can be classified according to the complexity 
of logical formulas expressing them. This leads to the most perspicu- 
ous and fully grammar independent characterization of tree families by 
monadic second-order logic. Although this characterization encompasses 
only regular tree sets the lifting process of the preceding section allows 
us to simulate the effect of macro-like productions with regular rewrite 
rules. 

Again, the device of lifting an alphabet into its derived form is not 
without its alternatives in terms of which a regular tree set can be created 
that has as value the intended set of tree structures over the original 
alphabet. Our own reason for resting with the lifting process was the 
need to carry through the "regularizing" interpretation not only for the 
generated language, but also for the derivation steps. 

A very simple example is now given of a context-free tree grammar that 
specifies as its frontier the (non-context-free) string language {a"6"c"}. 
At the same time we shall present the lifted version of the grammar and 
illustrate the effect of the productions with two sequences of derivation 
trees. 

Example 8. Consider the context-free tree grammar G — {J^, S, P) 
which consists of the components as shown below: 

So = {a,b,c} S2 = {^} 

To = {S} T3 = {F} 



P = 




b,c) 



ia,xi),^ib, X2),^ic,X3))\ 

{X1,X2),X3)) 



Applying the S'-production once and the first F-production two times 
we arrive at the sequence of trees in Figure |l|. 

The result of applying the terminal F-production to the last three trees 
in Figure ^ is shown in Figure ^. 

Transforming the grammar G with the help of the LIFT mapping of 
Definition ^ into its derived correspondent Gd produces a regular gram- 
mar. As will be recalled from the remarks after Definition 0, all sym- 
bols from the original alphabet become constant operators in the de- 
rived alphabet. In the presentation below the coding of type symbols is 
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-5 




Figure 1. An Example for {a"5"c"} 



taken over from the last example in section y. It relics upon the bijec- 
tion between S* x S and N, where 5 is a singleton. Let N be the set 
of sorts and let £>(S) be the derived alphabet. The derived grammar 
Gd = {D{T^), D{!F), D{S), D{P)) contains the following components: 





{a,6,c} 




- {S} 




= {^} 




= M 




= {S} 


^3 


= {F} 




■ S{F,a,b,c) 










K^3) 1 





[ SiF,S{-^,7r,,Sia)),S{'-,7r2,S{b)),S{'-,7r,,S{c))) J 

The context will always distinguish occurrences of the start symbol S 
from occurrences of the substitution operator S. 
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Figure 2. An Example of {a"&"c"} 



Sample derivations and two specimens of the generated language C{Gd) 
appear in Figures ^ and ^. 

The case illustrated by this example is characteristic of the general 
situation. An arbitrary context-free tree grammar G can be mapped into 
its derived counterpart with the help of the LIFT transformation. 
The result of this transformation process, Gd, is a regular grammar and 
therefore specifies a context-free language as the yield of its generated 
tree language C{Gd)- This follows directly from the fundamental fact, 
stated above, that a string language is context-free if and only if it is 
the leaf or frontier language of a regular tree language. The frontiers of 
C{G) and of C{G]j) are obviously not the same languages. The yield of 
C{Gjy) in particular consists of strings over the whole alphabet E extended 
by the set of projection symbols. Due to the fact, however, that the 
composition of the LIFT operation with the f3 operation is the identity on 
the elements of r(E, AT), it is of considerable interest to know whether 
this close relationship between elements of T{D{Y,) and of r(S],Ar) is 
preserved by the derivation process in the context-free grammar and its 
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Figure 4. Terminal 
example in Figure |^. 



derived trees corresponding to 
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regular counterpart. Before we prove a claim about this relationship in 
the proposition below a short historical remark appears to be apposite. 



The central theorem in Maibaum (1974) to the effect that every context 



free tree language is the image under the operation /? of an effectively con- 
structed regular language is wrong because he confounded the inside-out 
with the outside-in derivation mode. In the course of establishing a fixed- 
point characterization for context-free tree grammars in either generation 



mode Engclfriet and Schmidt (1977 ) point out this mistake and state as 



an immediate consequence of the fixed-point analysis of 10 context-free 
tree grammars within the space of the power-set tree substitution alge- 
bra that each 10 context-free tree language L is the image of a regular 
tree language D{C) under the unique homomorphism from T(Z?(S)) into 
£)T(I],X) (see their Cor. 4. 12). This immediate consequence is but a re- 
statement of the classical Mezei- Wright result that the equational subsets 
of an algebra are the homomorphic images of recognizable subsets in the 
initial term algebra. As formulated by Engelfriet & Schmidt, their correc- 
tion of Maibaum's theorem has a distinctive declarative content whereas 
the original claim had a clear operational meaning. It was based on 
the contention that the individual derivation steps of a context-free tree 
grammar and its derived counterpart correspond to each other. This is 
the point of the next lemma. 

Lemma Suppose G ~ {T,, S, P) is a context-free tree grammar and 
C{G) its generated tree language. Then there is a derived regular tree 
grammar Go — (DC^), Di^'), D{S), D{P)) such that C{G) is the image 
of C{Gd) under the unique homomorphism from the algebra of tree terms 
over Z?(E U J-) into the tree substitution algebra over the same alphabet. 
In particular, t' is derived in G from t in k steps, i.e. t ^ t' via the 
productions pi, . . . ,pk in P if and only if there are productions p'l, . . . , pj. 
in D{P) such that UFT{t') is derived in Gd from LIFT(t) via the corre- 
sponding productions. 

Proof. The proof is based on the closure of inside-out tree languages under 
tree homomorphisms. The idea of using the LIFT operation for the simula- 



tion of derivation st eps on the derived level can also be found in Engclfriet 
and Schmidt (19"78| ). Let /i„ be a family of mappings /i„ : E„ — > T{Q, X) 
where E and Q are two ranked alphabets. Such a family induces a tree 
homomorphism h : r(E) — > T{Q) according to the recursive stipulations: 

h{a) = ho{<7) for ct G Eg 

h{a{ti,...,tn)) = hn{a-)[h{ti), . . . ,h{tn)] for ct e E„ 
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A production p in P can be viewed as determining such a tree homomor- 
phism p : T{T, U J^) — > T{T, U J^) by considering the family of mappings 
p„ : 5]„UJ^„ — > T{'£UT,Xn) where p„(F) = i for t G T(Eu:r,X„) and 
Pn{f) = fi^i, ■ ■ ■ , Xn) for / 7^ F in S U JF. By requiring that p{xi) = Xi 
the mapping p can be regarded as a D(S U jr')-homomorphism from the 
tree substitution algebra X)T(I] U J-tX) into itself, where we have set 
T' :— J-\ {F}. By applying the LIFT-operation to the tree homomor- 
phism p we obtain its simulation po : T(Z)(S U J^)) — »■ T(£'(S U J^)) on 
the derived level: 

PD^F) = LIFT„(p„(F)) 

PDo{f) = f for/T^FinSU^ 

Observe, that we have treated D{T, U ^) as a ranked alphabet. If we can 
show that the diagram below commutes the claim in the lemma follows 
by induction: 

1{D{T,UJ^)) t(£)(su:p)) 

DT(SUjr,X) y 'Z)1{T,UJ^,X) 

P 

The commutability is shown by the succeeding series of equations in 
which the decisive step is justified by the identity of /3 o LIFT on the tree 
substitution algebra. Let / be in (E U J^)„: 

PiMf)) = /3(LIFT„(p„(/))) 

= Pnif) 

= P{f{xi,-.-,Xn)) 

= mm 

□ 

The preceding result provides an operational handle on the correspon- 
dence between the derivation sequences within the tree substitution alge- 
bra and the derived term algebra. As characterized, the correspondence 
is not of much help in finding a solution to the problem of giving a logical 
description of the exact computing power needed to analyze natural lan- 
guage phenomena. The formal definition of the /? transformation, which 
mediates the correspondence, is of an appealing perspicuity, but the many 
structural properties that are exemplified in the range of this mapping 
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make it difficult to estimate tlie computing resources necessary to estab- 
lish the correspondence relation between input trees and their values in 
the semantic domain of the tree substitution algebra. We know from the 
classical result for regular tree languages that monadic second-order logic 
is too weak to serve as a logical means to define the range of the (3 map- 
ping when it is applied to the space of regular tree languages. What does 
not seem to be excluded is the possibility of solving our logical charac- 
terization problem by defining the range of context-free tree languages 
within the domain of the regular languages. In this way, we would rest 
on firm ground and would take a glimpse into unknown territory by using 
the same logical instruments that helped us to survey our " recognizable" 
homeland. 



4. Logical Characterization 

Extending the characterization of grammatical properties by monadic 
second-order logic has been our main motivation. For tree languages the 
central result is the following: a tree language is definable in monadic 
second-order logic if and only if it is a regular language. As is well known 
and as we will indicate below, a similar characterization holds for regular 
and context-free string languages. The examples and analyses of English 



syntactic phenomena that are presented in Rogers (1994) make it abun- 
dantly clear that monadic second-order logic can be used as a flexible and 
powerful specification language for a wide range of theoretical constructs 
that form the core of one of the leading linguistic models. Therefore, the 
logical definability of structural properties that are coextensive with the 
empirically testified structural variation of natural languages, would be a 
useful and entirely grammar independent characterization of the notion 
of a possible human language. 

It follows from the cross-serial dependencies in Swiss German and re- 
lated phenomena in other languages that monadic second-order logic, at 
least in the form that subtends the characterization results just men- 
tioned, is not expressive enough to allow for a logical solution of the main 
problem of Universal Grammar: determining the range in cross-language 
variation. In many minds, this expressive weakness alone disqualifies 
monadic second-order logic from consideration in mctathcoretic studies 
on the logical foundations of linguistics. Employing the results of the 
preceding section, we will sketch a way out of this quandary, inspired by 
a familiar logical technique of talking in one structure about another. To 
what extent one can, based on this technique, simulate transformations 
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of trees is not yet fully understood. We shall sketch some further lines of 
research in the concluding remarks. 

The major aim of descriptive complexity theory consists in classify- 
ing properties and problems according to the logical complexity of the 
formulas in which they are expressible. One of the first results in this 
area of research is the definability of the classes of regular string and tree 
languages by means of monadic second-order logic. The use of this logic 
is of particular interest since it is powerful enough to express structural 
properties of practical relevance in the empirical sciences and it remains, 
in spite of its practical importance, efficiently solvable. As a preparation 
for our logical description of the (3 transformation we shall recall some 
concepts needed for expressing the logical characterization of the class of 
regular languages. 

For the purpose of logical definability we regard strings over a finite 
alphabet as model-theoretic structures of a certain kind. 

Definition 14. Let S be an alphabet and let t(S) be the vocabulary 
{<} U{Pa|aGS}, where < is binary and the Pa are monadic. A word 
model for u G S* is a structure of the form 

M = {B,<,Pa) 

where \B\ = length{u), < is an ordering of B and the Pa correspond to 
positions in u carrying the label a: 

Pa = {bG B\label{b) = a}. 

The corresponding monadic second-order logic MSOL(S) has node vari- 
ables x,y, z, . . . and set variables X,Y, . . . ranging over subsets of the 
domain of discourse B. There are four kinds of atomic formulas x = y, 
X < y, Pa{x) and x € X. The formulas of the language are constructed 
from the atomic formulas in the expected way by combining them by 
means of the usual propositional connectives and the existential and uni- 
versal quantifiers. Observe that not only individual node variables, but 
also node- set variables may be bound by quantifiers. For a closed formula 
A in this language and a word model M, we write |= A if A is true in 
M. The set of models that make a closed formula A true is denoted by 
Mod{A). A theorem by Elgot talks about the close relationship between 
model classes definable by closed formulas of MSOL(I]) and regular string 
languages. 

Theorem 1 (Elgot) . C is regular over the alphabet iff C is definable 

in monadic second-order logic over the vocabulary t(S). More succinctly: 

CeCs iff3A€ MSOL{i:) such that C = Mod{A) = {u G 12* \u \= A}. 



28 



UWE MONNICH 



For the statement of this classical result for tree languages we need an 
appropriate notion of model-theoretic structure. If S is a finite ranked 
alphabet, a tree in T(E) is coded by a model of the following form, which 
is called a labeled tree model. 

Definition 15. Let S be a finite ranked alphabet and let t(E) be the 
vocabulary {<i}i<n U {Pa \ a G S}, where each <, is binary and each Pa 
is a monadic predicate. A labeled tree model for f G Te is a structure 
of the form 

M = {B, <i, Pa)i<n,aeS 

where B is in one-one correspondence with dom{t), the tree domain of t, 

<i is the i-th successor function, n = the maximal m such that 7^ 
and the Pa correspond to positions with label a: 

Pa = {beB\ label{h) = a}. 

The notion of a labeled tree is built upon a system of nodes or tree 
addresses which are strings in N* and which satisfy two closure properties. 
More precisely, the domain of a finite ordered tree with at most n outgoing 
branches is a finite subset D C N* , where N„ = {0,...,n — 1}, such that 
the following conditions hold: 

(i) Ifuv € D {u, V e N* ) then u G D. 

(ii) If m e D and j < i then uj G D (m £ N* , i e N„). 

The appropriate monadic second-order logic of n successors MSOL„(S) 
has node variables x, y, z, . . . and set variables X,Y, Z, . . . , ranging over 
subsets of the tree domain. There are again four kinds of atomic formulas 
X = y, X <i y, Pa{x) and x G X, where we have presented the successor 
function in relational form. Arbitrary formulas are generated from atomic 
formulas by propositional connectives and the quantifiers binding both 
kinds of variables. If A is an MSOL„(S) sentence, i.e. a formula without 
free variables, and A4 a tree model, we write A4\=Ato indicate that A is 
true in M. If Mod{A) = {M | |= A}, we call this set of trees definable 
in MSOL„(S). The next theorem states the extension of Elgot's result to 
the yield or frontier of finite ranked ordered trees. 

Theorem 2 (Doner, Thatcher-Wright). Let T. be a ranked alphabet. 
a) £ C E* is context-free iff £ is the yield of a set of tree mod- 
els definable in monadic second-order logic of n successors over the 
vocabulary t(S). 

£ e £2 iff^A€ MSOLni^) such that £ = yield{Mod{A)) 
where Mod{A) = {t & Tj:\t \= A}. 
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b) A set of trees is generated by a regular tree grammar G iff there is 
a sentence A in MSO/.„(S) such that 

Mod{A) = C{G). 

Given the intimate relationship between regular tree languages and 
context-free string languages on the one hand and monadic second-order 
logic on the other, there is no hope to address the question as to how 
to account for context-sensitive formalisms with the definitional resources 
of this logic directly. An alternative approach is suggested by the char- 
acterization of the context-free tree languages as the class of languages 
that is the homomorphic image of the class of regular tree languages over 
a related alphabet. This characterization says that the context-free tree 
languages constitute the class of all tree families Tr{C) where £ is a sub- 
set of T{D{T,)) and Tr is a function from explicit trees over -D(S) to trees 
over S. 

The f3 transformation that sends regular tree languages to their context- 
free counterparts is a particular instance of a relation that mediates be- 
tween finite trees over different alphabets. If S and O are two alphabets, 
then a binary relation Tr C T(E) x T(rj) is generally called a tree trans- 
duction. Tree transductions can be interpreted as transforming a tree 
over one alphabet into a tree over another alphabet. Of special interest 
are those tree transductions that can be specified in an effective way and 
among these, the top-down and bottom-up transductions, performed by 
finite automata that process a tree from the root down or from the leaves 
up, have been thoroughly investigated since the early 70's. It has been 
shown that these types of transductions do not preserve regularity, but 
that their range is a proper subset of the context-free tree languages when 
their domain is restricted to the regular tree languages. Since the range 
of the (3 transformation is the full range of context-free tree languages 
under the same domain restriction its action cannot be simulated by a 
top-down or a bottom-up tree transduction. 

As the reader will recall, the transformations performed by the /? func- 
tion interpret the operators S and tt as substitution and projection, re- 
spectively. This behaviour is of such a formal simplicity that an effective 
description of its structural pattern should be possible, even though a 
simulation by a classical (top-down or bottom-up) tree automaton is ex- 
cluded according to the result just mentioned. Trees in T{D{Y^)) and trees 
in are really different ways of looking at one and the same thing. 

Therefore, it would further the project of an effective description of tree 
classes in the substitution algebra if there was an effective syntactic map 
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from the vocabulary -D(S) into the vocabulary S, because we know al- 
ready that monadic second-order logic provides an effective medium to 
define the regular tree classes in the initial _D(S)-algcbra. 

There is an established technique in model theory that enables one to 
show that in certain circumstances the choice of language is arbitrary. 
The central notion employed in this technique is that of a syntactic in- 
terpretation. The interesting feature of syntactic interpretation is that 
it allows one to talk in one structure about another, as we will now at- 
tempt to show. Suppose we have two rc^lational signatures E and Q with 
no constant or fuiic;tion symbols and. in addition to a domain formula 
Afi(x), for each n-ary i? G a defining formula Aii{xi, . . . ,Xn), where 
these formulas are built from the symbols in S. Given a S-structure A4 
in which the domain formula Aq{x) defines the universe of discourse of 
an fi-structure A4', it can then be shown that for every f2-formula B one 
can find an S-formula B' such that 

M\=B' [g] iSM'\=B [g] 

where g is a variable assignment in A4' . 

Note, that from a different perspective the domain formula Aq{x) can 
be viewed as establishing a binary relation between S- and f2-structures 
in the following way. We let Aq single out those E-structures where its 
satisfaction set is nonempty: 

M' = {a\M ^An[a]}^9. 

On M' an fi-structure is easily specified by the following stipulation for 
each R gQ: 

Rm' ■= {{ai,...,a„)\M \= ArIgi,. . . ,an] & 

M 1= Aaltti] for i = 1, . . . , n} 

(In case contains any functional relations appropriate closure conditions 
have to be required in addition to the non-emptiness of M' .) As defined, 
Ail is nothing but a binary relation Aq C Ms x Mo where Ms denotes 
the class of E-structurcs and M^ the class of il-structures: Two structures 
A4 and Ai' stand in the relation Aq if A1 is a E-structure in which we 
can talk about an internal 0-structure A4' that is carved out by the 
satisfaction of Aq. 

The above analysis involves the obvious suggestion to provide an ac- 
count of tree transductions within the model-theoretic framework of syn- 
tactic interpretation. Once labeled trees are looked at as specific model- 
theoretic structures a tree transduction Tr becomes a relation between 
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such structures in perfect analogy to the case of relational structures con- 
nected by a set of defining and domain formulas. 

The first to see the potential of applying the method of syntactic in- 
terpretation to the analysis of relations between graph-like structures was 
B. Courcellc. He introduces for this purpose the notion of a monadic 
second-order definable graph transduction. The switch from first-order 
logic to monadic second-order logic is caused by the same reasons that 
were given above for the adoption of monadic second-order logic in the 
case of labeled ordered trees. Apart from this recourse to higher-order 
logic Courcelle's definition is patterned upon the classical format pro- 
vided by the method of syntactic interpretation. Let us record here its 
adaptation to tree structures. 

Definition 16. Let S and fl be tow finite ranked alphabets. A monadic 
second-order definable tree transduction is defined as follows. We 

fix a syntactic interpretation I = {A, Aq{x), A^.{x,y), Ap^{x))i<n^aen 
consisting of a tuple of monadic second-order formulas over the signature 
E. These formulas are intended to define a tree model t' in T(f2) from a 
tree model t in T(E). The closed formula A is to define the domain of the 
transduction, the formula An{x) is to define the tree domain of t' and the 
formulas A^.{x,y) and Ap^{x) are to define the successor relations and 
the distribution of node labels on the tree domain dom{t'). n indicates 
the maximum arity of a symbol in f2. 

A tree model t' in T(ri) with tree domain D is defined in f e T(E) by 
/if 

a) (i) i h ^ 

(ii) D = {u\t^ An[u]} 

(iii) <,^={{u,v)(^D^\t^A^^ [u, v] } 

(iv) Pa^, ={u&D\t^ Ap^[u]} 

b) D satisfies the conditions of a tree domain with <i^, its successor 
relations. 

Indicating this last (functional) relation by defj [t) = t' we can finally say 
what is denoted by the monadic second-order tree transduction defined 
byL 

defi = {{t,t')\defi{t)=t'}. 

As defined, a definable tree transduction is indeed a tree transduction 
in the sense of the abstract transformation relation Tr C r(S) x T{il) be- 
tween tree domains that was introduced above. What is more, definable 
transductions possess the sort of computational properties that we asked 
of an effective syntactic mapping. The syntactic interpretation is a finitary 
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object, consisting of a tuple of formulas, and all the conditions in the def- 
inition are expressible by formulas of the monadic second-order language, 
which are decidable in general and even testable in linear time on given 
tree models. In other words, the co-domain of a definable transduction 
defi may not be definable itself, i.e. there may be no monadic second-order 
formula A' such that {t\t ^ A'} = {t' \ defi{t) = t' for some t s.t. t ^ A} 
where A defines the domain of de fi. The second-order theory of the co- 
domain of a definable transduction defj, on the other hand, is decidable 
and we can call it even definable if its domain formula is admitted as a 
restricting parameter. 

Definable tree transductions would thus provide a solution to the logi- 
cal characterization problem if the type of tree transformation performed 
by the /? mapping could be given a logical definition via a syntactic in- 
terpretation scheme. This is fortunately the case as stated in the next 
theorem. 

Theorem 3. The (3 transformation of a regular tree language is a monadic 
second- order definable tree translation. 

Proof Sketch. The detailed verification of the theorem's claim is left for 
another occasion. Let us point out that it can be reduced to the de- 
finability of the yield function of graph grammars with neighbourhood 
controlled embedding and dynamic edge relabeling (edNCE grammars). 
Productions in this type of grammar are of the form X — > {D, C) where 
X is an operative or functional node label, D is a. graph and C is a set 
of connection instructions. A derivation step licensed by such a produc- 
tion consists in replacing an X-labeled node by D and connecting D to 
the neighbourhood of X according to the specifications in C. As will be 
recalled, a much more specific form of local node replacement played a 
central role in macro-like productions of context-free tree grammars. It 
is straightforward to associate a derivation tree with a derivation in a ed- 
NCE grammar. A derivation is a labeled ordered tree in which the nodes 
are labeled with productions. If a node has the label X — > (D, C) its 
daughters are labeled by a sequence of production rules whose left-hand 
sides exhaust the operative nodes in the replacing graph D. The yield 
function executes the substitutions of the right-hand sides of the produc- 
tion rules in a bottom-up regime, observing along the way the connecting 
restrictions. The yield of the derivation tree returns as result the graph 
that is generated by a corresponding derivation of the edNCE grammar. 

The fact that the yield function of an edNCE grammar is definable in 
monadic second-order logic was proved by van Oostrom and presented in 
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a generalized form in Courcelle (1992). The trees in a regular tree gram- 
mar over a derived alphabet can be read as a special form of derivation 
trees in the sense of edNCE graph grammars. In these special trees, the 
replacements of the functional symbols have already been carried out, but 
the connecting instructions which are coded in the explicit substitution 
and projection labels have to be executed by the (3 transformations. Thus, 
the /? transformation turns out to be a special case of the yield function, 
which means that the f3 transformation can be simulated by a definable 
transduction. A detailed proof of the claim that the case of context-free 
tree languages is indeed covered by the context-free neighbourhood con- 
trolled graph languages will appear elsewhere. □ 



5. Concluding Remarks 

In the present paper, we have tried to combine the methods developed 
in the tradition of algebraic language theory with the techniques of model 
theory. While there is a perfect match between these two methodological 
frameworks in the case of regular or, equivalently, taking the automata- 
theoretic view, recognizable families of trees, this state of harmony is 
seriously disturbed once one considers the far richer family of context- 
free sets of trees. We had no choice but to take these richer structures 
into account, since a series of well documented non-local dependencies 
in natural languages fall outside the range of configurational properties 
that can be accommodated within the realm of regular trees. The gram- 
mar independent approach to characterizing syntactic concepts has been 
so rewarding, on the other hand, that it seemed worthwhile to make a 
serious attempt at providing the formal background to extend the tools 
of the logical analysis beyond the harmonious domain of strictly local 
dependencies. 

As was emphasized several times above, it may well be possible to 
strengthen the language of monadic second-order logic without losing its 
algorithmic tractability. Whether such a strengthening would create a 
framework for a direct application of monadic second-order logic's defi- 
nitional resources to the domain of context-free tree languages remains a 
challenging open problem. 

Deterred by the apparent obstacles to devising a manageable logical 
specification language for non-local dependencies we opted for the timid 
alternative, which consists in forming a picture of the outer world in terms 
of the conceptual network that fits so smoothly the regular homeland. 
This description of our approach refers both to the algebraic point of view 
according to which we evaluate (uniquely) the regular term expressions 
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in the semantic domain of context-free tree languages and to the logical 
point of view according to which we consider the recognizable forest of 
trees as relational structures in the model-theoretic sense and employ their 
language to talk about the foreign country where non-local dependencies 
roam. 

It is notoriously difficult to tell to what an extent the use of our own 
vernacular gives a distorted picture of a foreign country. The complex- 
ity of the clauses that enter into an explicit definition of the evaluation 
transducti(m would give an idea of how faithful our language reflects the 
situation in an area where it has not been used as a proven means of com- 
munication. We had to postpone the task of spelling out the translation 
scheme involved in such an evaluating transduction but the reader will be 
able to measure the distance between a syntactic configuration under its 
intended representation and its "regular" counterpart after a perusal of 
H.-P. Kolb's contribution to the present volume. 

There is a last disclaimer that needs to be made. The term definable 
transduction imports the misleading notion of an operational transforma- 
tion of structures. This misinterpretation would bestow a power upon the 
device of syntactic interpretation that it definitely does not have. The is- 
sue of whether certain linguistic phenomena are more easily described in a 
derivational rather than a representational model should not be confused 
with the issue of whether it is possible to give a direct logical definition of 
context-free tree structures or only an indirect definition via a definable 
transduction. The fact that we have opted for an indirect solution to our 
problem testifies only to our lack of ingenuity and not to our allegiance 
to the derivationists' cause. 
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